Exploratory Data Analysis

\(\hspace{0.3cm}\) More articles: \(\hspace{0.1cm}\) Estadistica4all

\(\hspace{0.3cm}\) Author: \(\hspace{0.1cm}\) Fabio Scielzo Ortiz

\(\hspace{0.3cm}\) If you use this article, please, reference it:

\(\hspace{0.5cm}\) Scielzo Ortiz, Fabio. (2023). Exploratory Data Analysis. http://estadistica4all.com/Articulos/EDA.html

It’s recommended to open the article on a computer or tablet.


1 Exploratory Data Analysis (EDA)

Exploratory data analysis (EDA) refers to the descriptive statistical analysis of a data-set.

Next we are going to propose a methodology to carry out an EDA, using Python as programming lenguage.

2 First step: data-set structure

2.1 Import data-set

First of all, we import the data-set with which we will work.

import pandas as pd

Netflix_Data = pd.read_csv('titles.csv')
Netflix_Data
id title type description release_year age_certification runtime genres production_countries seasons imdb_id imdb_score imdb_votes tmdb_popularity tmdb_score
0 ts300399 Five Came Back: The Reference Films SHOW This collection includes 12 World War II-era p… 1945.0 TV-MA 51 [‘documentation’] [‘US’] 1.0 NaN NaN NaN 0.600 NaN
1 tm84618 Taxi Driver MOVIE A mentally unstable Vietnam War veteran works … 1976.0 R 114 [‘drama’, ‘crime’] [‘US’] NaN tt0075314 8.2 808582.0 40.965 8.179
2 tm154986 Deliverance MOVIE Intent on seeing the Cahulawassee River before… 1972.0 R 109 [‘drama’, ‘action’, ‘thriller’, ‘european’] [‘US’] NaN tt0068473 7.7 107673.0 10.010 7.300
3 tm127384 Monty Python and the Holy Grail MOVIE King Arthur, accompanied by his squire, recrui… 1975.0 PG 91 [‘fantasy’, ‘action’, ‘comedy’] [‘GB’] NaN tt0071853 8.2 534486.0 15.461 7.811
4 tm120801 The Dirty Dozen MOVIE 12 American military prisoners in World War II… 1967.0 NaN 150 [‘war’, ‘action’] [‘GB’, ‘US’] NaN tt0061578 7.7 72662.0 20.398 7.600
5845 tm1014599 Fine Wine MOVIE A beautiful love story that can happen between… 2021.0 NaN 100 [‘romance’, ‘drama’] [‘NG’] NaN tt13857480 6.8 45.0 1.466 NaN
5846 tm898842 C/O Kaadhal MOVIE A heart warming film that explores the concept… 2021.0 NaN 134 [‘drama’] [] NaN tt11803618 7.7 348.0 NaN NaN
5847 tm1059008 Lokillo MOVIE A controversial TV host and comedian who has b… 2021.0 NaN 90 [‘comedy’] [‘CO’] NaN tt14585902 3.8 68.0 26.005 6.300
5848 tm1035612 Dad Stop Embarrassing Me - The Afterparty MOVIE Jamie Foxx, David Alan Grier and more from the… 2021.0 PG-13 37 [] [‘US’] NaN NaN NaN NaN 1.296 10.000
5849 ts271048 Mighty Little Bheem: Kite Festival SHOW With winter behind them, Bheem and his townspe… 2021.0 NaN 7 [‘family’, ‘animation’, ‘comedy’] [] 1.0 tt13711094 7.8 18.0 2.289 10.000

5850 rows × 15 columns


2.2 Data-set conceptual description

This data-set has information about 15 variables on 5850 Netflix titles.

Next table has a brief conceptual description about data-set variables:

Variable Descripción Tipo
id The title ID on JustWatch Identifier
title The name of the title Text
type TV show or movie Categorical
description A brief description Text
release_year release year Quantitative
age_certification age rating Categorical
runtime the number of episodes (show), the duration time in minutes (movie) Quantitative
genres A list of genres Categorical
production_countries A list of countries that produced the title Categorical
seasons Number of seasons if it’s a SHOW Quantitative
imdb_id The title ID on IMDB Identifier
imdb_score Rating on IMDB Quantitative
imdb_votes number of votes on IMDB Quantitative
tmdb_popularity Popularity on TMDB Quantitative
tmdb_score Rating on TMDB Quantitative


2.3 Data-set size

We can get the data-set size as the number of rows and columns of the data-set.

Netflix_Data.shape
(5850, 15)

As discussed above, the data-set has 5850 rows and 15 columns.


2.4 info() method

Netflix_Data.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5850 entries, 0 to 5849
Data columns (total 15 columns):
 #   Column                Non-Null Count  Dtype  
---  ------                --------------  -----  
 0   id                    5850 non-null   object 
 1   title                 5849 non-null   object 
 2   type                  5850 non-null   object 
 3   description           5832 non-null   object 
 4   release_year          5850 non-null   int64  
 5   age_certification     3231 non-null   object 
 6   runtime               5850 non-null   int64  
 7   genres                5850 non-null   object 
 8   production_countries  5850 non-null   object 
 9   seasons               2106 non-null   float64
 10  imdb_id               5447 non-null   object 
 11  imdb_score            5368 non-null   float64
 12  imdb_votes            5352 non-null   float64
 13  tmdb_popularity       5759 non-null   float64
 14  tmdb_score            5539 non-null   float64
dtypes: float64(5), int64(2), object(8)
memory usage: 685.7+ KB

info() method give us column names, number of non null values in each column and column type.


2.5 Column types

Netflix_Data.dtypes
id                       object
title                    object
type                     object
description              object
release_year              int64
age_certification        object
runtime                   int64
genres                   object
production_countries     object
seasons                 float64
imdb_id                  object
imdb_score              float64
imdb_votes              float64
tmdb_popularity         float64
tmdb_score              float64
dtype: object

It’s another way to get column types.

Object is the typical type of categorical variables, identifier or text.

Float64 and int64 is the typical type of quantitative variables, float64 for continuous one, and int64 for discrete one.


2.6 Change column types

We can change the type of a column with astype() method:

Netflix_Data['release_year'] = Netflix_Data['release_year'].astype('float')

We can check if the changes have been done correctly:

Netflix_Data.dtypes
id                       object
title                    object
type                     object
description              object
release_year            float64
age_certification        object
runtime                   int64
genres                   object
production_countries     object
seasons                 float64
imdb_id                  object
imdb_score              float64
imdb_votes              float64
tmdb_popularity         float64
tmdb_score              float64
dtype: object


2.7 NaN identification

A NaN is a not a number value. NaN is equivalent to missing value.

We are going to calculate, for each variable, the proportion of missing values over the total number of observations:

Prop_NA = Netflix_Data.isnull().sum() / len(Netflix_Data)

Prop_NA
id                      0.000000
title                   0.000171
type                    0.000000
description             0.003077
release_year            0.000000
age_certification       0.447692
runtime                 0.000000
genres                  0.000000
production_countries    0.000000
seasons                 0.640000
imdb_id                 0.068889
imdb_score              0.082393
imdb_votes              0.085128
tmdb_popularity         0.015556
tmdb_score              0.053162
dtype: float64


We can see that there are variables with a high proportion of missing values, as age_certification (44.77%).

season would be the variable with higher proportion of missing values, but it is because of season only is defined for type=SHOW.


3 Auxiliar step: data preprocessing

3.1 Categorization of cuantitative variables

3.2 Dummification of categorical variables

3.3 Recodefication of categorical variables

3.4 Variable Scaling

3.5 Dealing with NaN


4 Second step: statistical description

4.1 Statistical variable

A statistical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\) can be modeled as a random variable.

Under this approach, we can apply all probability theory on random variables to statistical variables. \(\\[0.4cm]\)

4.2 Range of a statistical variable

The range of a statistical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\) is denoted by \(\hspace{0.05cm}Range(\mathcal{X}_k)\hspace{0.05cm}\), and is defined as the set of possible values of \(\hspace{0.05cm}\mathcal{X}_k\). \(\\[0.4cm]\)

4.2.1 Statistical variable types: quantitative and categorical

  • The variable \(\mathcal{X}_k\) is quantitative if the elements of it´s range are conceptually numbers. \(\\[0.5cm]\)

  • The variable \(\mathcal{X}_k\) is categorical if the elements of it´s range aree labels or categories (they can be numbers at a symbolic level but not at a conceptual level). \(\\[0.4cm]\)

4.2.2 Quantitative variable types: continuous and discrete

We can distinguish at least two types of quantitative variables: continuous and discrete.

  • \(\mathcal{X}_k\hspace{0.05cm}\) is continuous if \(\hspace{0.05cm}Range(\mathcal{X}_k)\hspace{0.05cm}\) is a not countable set. \(\\[0.5cm]\)

  • \(\mathcal{X}_k\hspace{0.05cm}\) is discrete if \(\hspace{0.05cm}Range(\mathcal{X}_k)\hspace{0.05cm}\) is countable set. \(\\[0.2cm]\)

Note:

In particular, variables whose range is a finite set will be discrete.

Variables whose range isn´t a finite set will be continuous. \(\\[0.4cm]\)

4.2.3 Categorical variable types: r-ary

Let \(\mathcal{X}_k\) a categorical variable ,

  • \(\mathcal{X}_k\) is r-aria if it´s range has r elements that are categories or labels.

In Statistics binary (2-aria) categorical variables are particularly important. \(\\[0.4cm]\)

4.2.4 Categorical variable types: nominal and ordinal

Let \(\mathcal{X}_k\) a \(r\)-ary categorical variable.

  • \(\mathcal{X}_k\) is nominal if there is no ordering between the \(r\) categories of it’s range. \(\\[0.4cm]\)

  • \(\mathcal{X}_k\) is ordinal if there is ordering between the \(r\) categories of it’s range. \(\\[0.4cm]\)

4.3 Sample of a statistical variable

Given a statistical variable \(\hspace{0.05cm}\mathcal{X}_k\).

A sample of \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\) is a vector of values of \(\hspace{0.05cm}\mathcal{X}_k\), called observations.

Therefore:

\[ X_k \hspace{0.05cm} = \hspace{0.05cm} \begin{pmatrix} x_{1k} \\ x_{2k}\\ ... \\ x_{nk} \end{pmatrix} \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t \\ \]

is a sample of a statistical variable because is a vector with the values or observations of the variable \(\hspace{0.05cm} \mathcal{X}_k \hspace{0.05cm}\) for \(\hspace{0.05cm} n \hspace{0.05cm}\) elements or individuals of a sample.

Where: \(\hspace{0.1cm} x_{ik}\hspace{0.05cm}\) is the value \(\hspace{0.05cm} i\)-th observation of the variable \(\hspace{0.05cm} \mathcal{X}_k\). \(\\[0.4cm]\)

4.4 Arithmetic Mean

Given a quantitative statistical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

The arithmetic mean of \(\hspace{0.05cm}X_k \hspace{0.05cm}\) is defined as: \(\\[0.3cm]\)

\[\overline{\hspace{0.05cm} X_k \hspace{0.05cm} } \hspace{0.1cm}=\hspace{0.1cm} \dfrac{1}{n} \cdot \sum_{i=1}^n \hspace{0.05cm} x_{ik}\] \(\\[0.4cm]\)

4.5 Weighted Mean

Given a quantitative statistical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

And given a weights for each observation of the variable \(\hspace{0.05cm} \mathcal{X}_k \hspace{0.2cm} \Rightarrow \hspace{0.2cm}\) \(w \hspace{0.05cm} = \hspace{0.05cm} (w_1,w_2,...,w_n)^t\)

The weighted mean of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) with the weights vector \(\hspace{0.05cm} w \hspace{0.05cm}\) is defined as:

\[ \overline{X_k} (w) \hspace{0.1cm} = \hspace{0.1cm} \dfrac{1}{\hspace{0.1cm}\sum_{i=1}^{n} \hspace{0.05cm} w_{i} \hspace{0.1cm}} \hspace{0.05cm}\cdot\hspace{0.05cm} \sum_{i=1}^{n} \hspace{0.1cm} x_{ik} \cdot w_i \] \(\\[0.4cm]\)

4.6 Geometric Mean

Given the variable \(\hspace{0.05cm} X_k=(x_{1k}, x_{2k},...,x_{nk})^t\).

The geometric mean of the variable \(\hspace{0.05cm}X_k\hspace{0.05cm}\) is defined as: \(\\[0.3cm]\)

\[ \overline{X_k}_{geo} \hspace{0.05cm} = \hspace{0.05cm} \sqrt{\Pi_{i=1}^{n} x_{ik}} \hspace{0.05cm} = \hspace{0.05cm} \sqrt{x_{1k}\cdot x_{2k}\cdot...\cdot x_{nk}} \] \(\\[0.4cm]\)

4.7 Median

Given a statistical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

The median of \(\hspace{0.05cm}X_k \hspace{0.05cm}\) is defined as a value \(Me(X_k)\) such that: \(\\[0.3cm]\)

\[\dfrac{1}{n} \cdot \sum_{i=1}^n \hspace{0.1cm} \mathbb{I} \hspace{0.05cm} \bigl[ \hspace{0.1cm} x_{ik} \hspace{0.05cm} \leq \hspace{0.05cm} Me(X_k) \hspace{0.1cm} \bigr] \hspace{0.1cm} = \hspace{0.1cm} 0.50\]

where: \(\hspace{0.15cm}\mathbb{I}\hspace{0.1cm}\) is the indicator function. \(\\[0.4cm]\)

4.8 Mode

Given a categorical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

The mode of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) is the most repeated value in \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\), so, the mode of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) is the most frequent value of \(\hspace{0.05cm} X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) \(\\[0.4cm]\)

4.9 Variance

Given a quantitative variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

The variance of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) is defined as:

\[\sigma(X_k)^2 \hspace{0.1cm} = \hspace{0.1cm} \dfrac{1}{n} \cdot \sum_{i=1}^n \hspace{0.05cm} \left(\hspace{0.05cm} x_{ik} - \overline{X_k} \hspace{0.05cm}\right)^2\]

The standard deviation or standard error of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) is defined as:

\[\sqrt{ \sigma(X_k)^2 } \hspace{0.1cm} = \hspace{0.1cm} \dfrac{1}{n} \cdot \sum_{i=1}^n \left( \hspace{0.05cm} x_{ik} - \overline{X_k} \hspace{0.05cm} \right)\] \(\\[0.4cm]\)

4.10 Median Absolute Deviation

Given a statistical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

The median absolute deviation (MAD) of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) is defined as:

\[MAD(X_k) \hspace{0.1cm} = \hspace{0.1cm} Me \bigl( \hspace{0.1cm} \left| \hspace{0.05cm} X_k - Me(X_k) \hspace{0.05cm} \right| \hspace{0.1cm} \bigr) \hspace{0.1cm} = \hspace{0.1cm} Me \hspace{0.1cm} \Bigr[ \hspace{0.1cm} \left( \hspace{0.2cm} \left| \hspace{0.1cm} x_{ik} - Me(X_k) \hspace{0.1cm} \right| \hspace{0.15cm} : \hspace{0.15cm} i = 1,\dots,n \hspace{0.2cm} \right) \hspace{0.1cm} \Bigr]\] \(\\[0.4cm]\)

4.11 Quantiles

Given a statistical variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

The \(\hspace{0.05cm}q\)-order quantile of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) is defined as a value \(Q(X_k , q)\) such that:

\[\dfrac{1}{n} \cdot \sum_{i=1}^n \hspace{0.1cm} \mathbb{I} \hspace{0.05cm} \bigl[ \hspace{0.1cm} x_{ik} \hspace{0.05cm} \leq \hspace{0.05cm} Q(\hspace{0.05cm} X_k \hspace{0.05cm},\hspace{0.05cm} q \hspace{0.05cm}) \hspace{0.1cm} \bigr] \hspace{0.1cm} = \hspace{0.1cm} q\]

where: \(\hspace{0.15cm}\mathbb{I}\hspace{0.1cm}\) is the indicator function. \(\\[0.3cm]\)

Observation:

The median is the 0.5-order quantile. \(\\[0.4cm]\)

4.12 Kurtosis

Given a quantitative variable \(\hspace{0.05cm}\mathcal{X}_k\hspace{0.05cm}\), and a sample \(\hspace{0.05cm}X_k \hspace{0.05cm} = \hspace{0.05cm} \left( \hspace{0.01cm} x_{1k} \hspace{0.01cm} , \hspace{0.01cm}x_{2k}\hspace{0.01cm},\dots ,\hspace{0.01cm} x_{nk} \hspace{0.01cm}\right)^t\hspace{0.05cm}\) of that statistical variable.

The kurtosis coefficient of \(\hspace{0.05cm} X_k \hspace{0.05cm}\) is defined as:

\[ \Psi_X = \dfrac{\mu_{4}}{\sigma^{4}} \]

where:

\[ \mu_{4}=\frac{1}{n}\sum_{i=1}^{n} (X_i)^{4} \\ \sigma = \sqrt{Var(X)} \\ \]

Propierties:

\end{gather*} Si \(\Psi_X > 3\) \(\Rightarrow\) la distribucion de X es mas apuntada y con colas mas gruesas que la normal.

Si \(\Psi_X < 3\) \(\Rightarrow\) la distribucion de X es menos apuntada y con colas menos gruesas que la normal.

4.13 Skewness

Sea X variable numerica de la que se tiene una muestra de datos \(\lbrace\) \(X_i\) / \(i=1,...,n\) \(\rbrace\) ,

El coeficiente de asimetria de Fisher mide el grado de simetria de la distribucion de unos datos respecto de su media aritmetica.

Si \(\gamma_X > 0\) \(\Rightarrow\) la distribución de X es asimétrica positiva o a la derecha.

Si \(\gamma_{X} <0\) \(\Rightarrow\) la distribución es asimétrica negativa o a la izquierda.

Si la distribución es simétrica respecto de la media \(\Rightarrow\) \(\gamma_{X}=0\) . El recíproco no es cierto.

4.14 Outliers

4.15 Data matrix

Data Martrix Representation by Rows Data Martrix Representation by columns

4.16 Covariance

4.17 Correlation

4.18 Statistical description for quantitative variables

mean. median, variance, cuantiles, kurtosis, skewness, outliers

frequency tables –> https://www.statology.org/frequency-tables-python/

4.19 Statistical description for categorical variables

mode, quantiles

frequency tables

4.20 Statistical description for variable crossings (cruces de variables cuantis-categoricas, categroicas-categoricas, cuantis-cuantis)

quantitative-categorical –> mean, median, vaariance, quantiles etc BY GROUPS. Joint and conditional frequency tables.

categorical-categorical –> Joint and conditional frequency tables.

quantitative-quantitative –> transform to categorical-categorical case.

5 Third step: statistical visualization

5.1 Visualization for quantitative variables

5.2 Visualization for categorical variables

5.3 Visualization for quantitative-categorical

5.4 Visualization for categorical-categorical


6 Descripción Estadistica Básica

A continuación vamos a realizar una descripción estadistica básica de las variables, a traves de dicersos estadisticos básicos.

6.1 Estadisticos básicos para las variables cuantitativas

Para las variables cuantitativas:

Netflix_Data.describe()
release_year runtime seasons imdb_score imdb_votes tmdb_popularity tmdb_score
count 5850.000000 5850.000000 2106.000000 5368.000000 5.352000e+03 5759.000000 5539.000000
mean 2016.417094 76.888889 2.162868 6.510861 2.343938e+04 22.637925 6.829175
std 6.937726 39.002509 2.689041 1.163826 9.582047e+04 81.680263 1.170391
min 1945.000000 0.000000 1.000000 1.500000 5.000000e+00 0.009442 0.500000
25% 2016.000000 44.000000 1.000000 5.800000 5.167500e+02 2.728500 6.100000
50% 2018.000000 83.000000 1.000000 6.600000 2.233500e+03 6.821000 6.900000
75% 2020.000000 104.000000 2.000000 7.300000 9.494000e+03 16.590000 7.537500
max 2022.000000 240.000000 42.000000 9.600000 2.294231e+06 2274.044000 10.000000


6.2 Estadisticos básicos para las variables categóricas

Para las variables categóricas (no cuantitativas, en general) :

Netflix_Data.loc[: , ['title', 'description', 'age_certification', 'genres', 'production_countries' ]].describe()
title description age_certification genres production_countries
count 5849 5832 3231 5850 5850
unique 5798 5829 11 1726 452
top The Gift Five families struggle with the ups and downs … TV-MA [‘comedy’] [‘US’]
freq 3 2 883 484 1959


6.3 Gráficos conjuntos para las variables cuantitativas

En esta seccion vamos a hacer un analisis gráfico básico de las variables cuantitativas, consideradas de manera conjunta.

Cargamos las librerias necesarias para los gráficos:

import numpy as np

import seaborn as sns
import matplotlib as mpl
import matplotlib.pyplot as plt

6.3.1 Histograma conjunto de las variables cuantitativas

Vamos a generar un grafico con un histograma para cada una de las variables cuantitativas:

fig, axs = plt.subplots(3, 3, figsize=(11, 11))

p1 = sns.histplot(data=Netflix_Data, x="release_year", stat="proportion", bins=15, color="skyblue", ax=axs[0, 0])
 

p2 = sns.histplot(data=Netflix_Data, x="runtime", stat="proportion", bins=15, color="olive", ax=axs[0, 1])
p2.axes.set(xlabel='runtime', ylabel=' ')
 

p3 = sns.histplot(data=Netflix_Data, x="seasons", stat="proportion", bins=15, color="blue", ax=axs[0, 2])
p3.axes.set(xlabel='seasons', ylabel=' ')
 

p4 = sns.histplot(data=Netflix_Data, x="imdb_score", stat="proportion", bins=15, color="teal", ax=axs[1, 0])
p4.axes.set(xlabel='imdb_score', ylabel=' ')
 

p5 = sns.histplot(data=Netflix_Data, x="imdb_votes", stat="proportion", bins=15, color="purple", ax=axs[1, 1])
p5.axes.set(xlabel='imdb_votes', ylabel=' ')
 

p6 = sns.histplot(data=Netflix_Data, x="tmdb_popularity", stat="proportion", bins=15, color="pink", ax=axs[1, 2])
p6.axes.set(xlabel='tmdb_popularity', ylabel=' ')
 
 
p7 = sns.histplot(data=Netflix_Data, x="tmdb_score", stat="proportion", bins=15, color="red", ax=axs[2, 0])
p7.axes.set(xlabel='tmdb_score', ylabel=' ')
 
fig.savefig('p1.png', format='png', dpi=1200)

plt.show()


Histograma conjunto de las variables cuantitativas


6.3.2 Box-Plot conjunto de las variables cuantitativas

Vamos a generar un grafico con un box-plot para cada una de las variables cuantitativas:

fig, axs = plt.subplots(3, 3, figsize=(11, 11))

p1 = sns.boxplot(data=Netflix_Data, x="release_year", color="skyblue", ax=axs[0, 0])
 

p2 = sns.boxplot(data=Netflix_Data, x="runtime",  color="olive", ax=axs[0, 1])
p2.axes.set(xlabel='runtime', ylabel=' ')
p2.set_xticks( range(int(Netflix_Data['runtime'].min()) , int(Netflix_Data['runtime'].max()) , 100) )
p2.set_yticks( np.arange(0, 1, 0.1)  )

p3 = sns.boxplot(data=Netflix_Data, x="seasons", color="blue", ax=axs[0, 2])
p3.axes.set(xlabel='seasons', ylabel=' ')
 

p4 = sns.boxplot(data=Netflix_Data, x="imdb_score", color="teal", ax=axs[1, 0])
p4.axes.set(xlabel='imdb_score', ylabel=' ')
p4.set_xticks( range(int(Netflix_Data['imdb_score'].min()) , int(Netflix_Data['imdb_score'].max()) , 300) )
p4.set_yticks( np.arange(0, 1, 0.1)  )

p5 = sns.boxplot(data=Netflix_Data, x="imdb_votes", color="purple", ax=axs[1, 1])
p5.axes.set(xlabel='imdb_votes', ylabel=' ')
p5.set_xticks( range(int(Netflix_Data['imdb_votes'].min()) , int(Netflix_Data['imdb_votes'].max()/2) , 500000) )
p5.set_yticks( np.arange(0, 1, 0.1)  )

p6 = sns.boxplot(data=Netflix_Data, x="tmdb_popularity", color="pink", ax=axs[1, 2])
p6.axes.set(xlabel='tmdb_popularity', ylabel=' ')
p6.set_xticks( range(int(Netflix_Data['tmdb_popularity'].min()) , int(Netflix_Data['tmdb_popularity'].max()+1) , 1000) )
p6.set_yticks( np.arange(0, 1, 0.1)  )
 
p7 = sns.boxplot(data=Netflix_Data, x="tmdb_score", color="red", ax=axs[2, 0])
p7.axes.set(xlabel='tmdb_score', ylabel=' ')
p7.set_xticks( range(int(Netflix_Data['tmdb_score'].min()) , int(Netflix_Data['tmdb_score'].max()+1) , 2) )
p7.set_yticks( np.arange(0, 1, 0.1)  )

plt.show()


Box-Plot conjunto de las variables cuantitativas


6.3.3 Empirical-Cumulative-Distribution-Function-Plot conjunto de las variables cuantitativas

Vamos a generar un grafico con un ECDF-plot para cada una de las variables cuantitativas:

fig, axs = plt.subplots(3, 3, figsize=(11, 11))

p1 = sns.ecdfplot(data=Netflix_Data, x="release_year", color="skyblue", ax=axs[0, 0])
p1.set_xticks( range(int(Netflix_Data['release_year'].min()) , int(Netflix_Data['release_year'].max()+20) , 20) )
p1.set_yticks( np.arange(0, 1, 0.1)  )

p2 = sns.ecdfplot(data=Netflix_Data, x="runtime",  color="olive", ax=axs[0, 1])
p2.axes.set(xlabel='runtime', ylabel=' ')
p2.set_xticks( range(int(Netflix_Data['runtime'].min()) , int(Netflix_Data['runtime'].max()) , 100) )
p2.set_yticks( np.arange(0, 1, 0.1)  )

p3 = sns.ecdfplot(data=Netflix_Data, x="seasons", color="blue", ax=axs[0, 2])
p3.axes.set(xlabel='seasons', ylabel=' ')
p3.set_xticks( range(int(Netflix_Data['seasons'].min()) , int(Netflix_Data['seasons'].max()) , 4) )
p3.set_yticks( np.arange(0, 1, 0.1)  )

p4 = sns.ecdfplot(data=Netflix_Data, x="imdb_score", color="teal", ax=axs[1, 0])
p4.axes.set(xlabel='imdb_score', ylabel=' ')
p4.set_xticks( range(int(Netflix_Data['imdb_score'].min()) , int(Netflix_Data['imdb_score'].max()) , 300) )
p4.set_yticks( np.arange(0, 1, 0.1)  )

p5 = sns.ecdfplot(data=Netflix_Data, x="imdb_votes", color="purple", ax=axs[1, 1])
p5.axes.set(xlabel='imdb_votes', ylabel=' ')
p5.set_xticks( range(int(Netflix_Data['imdb_votes'].min()) , int(Netflix_Data['imdb_votes'].max()/2) , 500000) )
p5.set_yticks( np.arange(0, 1, 0.1)  )

p6 = sns.ecdfplot(data=Netflix_Data, x="tmdb_popularity", color="pink", ax=axs[1, 2])
p6.axes.set(xlabel='tmdb_popularity', ylabel=' ')
p6.set_xticks( range(int(Netflix_Data['tmdb_popularity'].min()) , int(Netflix_Data['tmdb_popularity'].max()+1) , 1000) )
p6.set_yticks( np.arange(0, 1, 0.1)  )
 
p7 = sns.ecdfplot(data=Netflix_Data, x="tmdb_score", color="red", ax=axs[2, 0])
p7.axes.set(xlabel='tmdb_score', ylabel=' ')
p7.set_xticks( range(int(Netflix_Data['tmdb_score'].min()) , int(Netflix_Data['tmdb_score'].max()+1) , 50) )
p7.set_yticks( np.arange(0, 1, 0.1)  )

plt.show()


ECDF-Plot conjunto de las variables cuantitativas


6.4 Gráficos conjuntos para las variables categoricas

6.4.1 Bar-plot conjunto de las variables categóricas

Vamos a generar un grafico con un bar-plot para cada una de las variables categóricas, excepto para aquellas cuyo nº de categorias es excesivo, y por tanto no es práctico el gráfico:

fig, axs = plt.subplots(1, 2, figsize=(13, 6))

p1 = sns.countplot(x='type', data=Netflix_Data, ax=axs[0]) 
p1.set_xticklabels(['Movie', 'Show'])
p1.axes.set(xlabel='type', ylabel='count')

p2 = sns.countplot(x='age_certification', data=Netflix_Data, ax=axs[1]) 

plt.show()


Bar-Plot conjunto de variables categoricas


7 Análisis Estadístico

En la sección anterior se hizo una descripción estadistica básica de las variables del data-set con el que estamos trabajando, pero no se ha hecho ningun analisis de los resultados obtenidos.

En esta seccion además de ampliar la descripción estadistica de los datos, se llevará a cabo un analisis de los resultados obtenidos.


7.1 Estadisticos para las variables cuantitativas

7.2 Estadisticos para las variables categoricas

7.3 Tablas de frecuencias

7.4 Gráficos individuales para las variables cuantitativas

7.5 Gráficos individuales para las variables categoricas